Search for: All records

Background and Objectives: The variability and biases in the real-world performance benchmarking of deep learning models for medical imaging compromise their trustworthiness for real-world deployment. The common approach of holding out a single fixed test set fails to quantify the variance in the estimation of test performance metrics. This study introduces NACHOS (Nested and Automated Cross-validation and Hyperparameter Optimization using Supercomputing) to reduce and quantify the variance of test performance metrics of deep learning models. Methods: NACHOS integrates Nested Cross-Validation (NCV) and Automated Hyperparameter Optimization (AHPO) within a parallelized high-performance computing (HPC) framework. NACHOS was demonstrated on a chest X-ray repository and an Optical Coherence Tomography (OCT) dataset under multiple data partitioning schemes. Beyond performance estimation, DACHOS (Deployment with Automated Cross-validation and Hyperparameter Optimization using Supercomputing) is introduced to leverage AHPO and cross-validation to build the final model on the full dataset, improving expected deployment performance. Results: The findings underscore the importance of NCV in quantifying and reducing estimation variance, AHPO in optimizing hyperparameters consistently across test folds, and HPC in ensuring computational feasibility. Conclusions: By integrating these methodologies, NACHOS and DACHOS provide a scalable, reproducible, and trustworthy framework for DL model evaluation and deployment in medical imaging. To maximize public availability, the full open-source codebase is provided at https://github.com/thepanlab/NACHOS. 

ABSTRACT A three‐dimensional convolutional neural network (3D‐CNN) was developed for the analysis of volumetric optical coherence tomography (OCT) images to enhance endoscopic guidance during percutaneous nephrostomy. The model was performance‐benchmarked using a 10‐fold nested cross‐validation procedure and achieved an average test accuracy of 90.57% across a dataset of 10 porcine kidneys. This performance significantly exceeded that of 2D‐CNN models that attained average test accuracies ranging from 85.63% to 88.22% using 1, 10, or 100 radial sections extracted from the 3D OCT volumes. The 3D‐CNN (~12 million parameters) was benchmarked against three state‐of‐the‐art volumetric architectures: the 3D Vision Transformer (3D‐ViT, ~45 million parameters), 3D‐DenseNet121 (~12 million parameters), and the Multi‐plane and Multi‐slice Transformer (M3T, ~29 million parameters). While these models achieved comparable inferencing accuracy, the 3D‐CNN exhibited lower inference latency (33 ms) than 3D‐ViT (86 ms), 3D‐DenseNet121 (58 ms), and M3T (93 ms), representing a critical advantage for real‐time surgical guidance applications. These results demonstrate the 3D‐CNN's capability as a powerful and practical tool for computer‐aided diagnosis in OCT‐guided surgical interventions. 

Kidney transplantation remains the preferred treatment for patients with end-stage kidney disease. However, the ongoing shortage of donor organs continues to limit the availability of transplant treatments. Existing evaluation methods, such as the kidney donor profile index (KDPI) and pre-transplant donor biopsy (PTDB), have various limitations, including low discriminative power, invasiveness, and sampling errors, which reduce their effectiveness in organ quality assessment and contribute to the risk of unnecessary organ discard. In this study, we explored the dynamic optical coherence tomography (DOCT) as a label-free, non-invasive approach to monitor the viability ofex vivomouse kidneys during static cold storage over 48 hours. The dynamic metrics logarithmic intensity variance (LIV), early OCT correlation decay speed (OCDS_e), and late OCT correlation decay speed (OCDS_l) were extracted from OCT signal fluctuations to quantify temporal and spatial tissue activity and deterioration. Our results demonstrate that DOCT provides complementary information relevant to tissue viability, in addition to the morphological assessment offered by conventional OCT imaging, showing potential to improve pre-transplant organ evaluation and clinic decision-making.